We manually curated 500\(+\) multiple modality PPMI MRI subjects. 5 messages from this effort.
failure rates humans vs processed
we provide scientists handles on this so that can be “strict” or “loose” in data selection
we document these parameters
for algorithm developers
for scientists
Enables investigation of the “meaning” of M3RI at the individual level
Empowers investigation of the “meaning” of M3RI at the population level
Reviewing multiple modality data quality for current phase of collection
we collaborated on cohort definition
first phase - select subjects based on sampling from sites
second phase - select subjects based on population characteristics (MJFF/PPMI)
show table of summary cohort
| p | ||||||
|---|---|---|---|---|---|---|
| (N=47) | (N=1) | (N=4) | (N=230) | (N=217) | ||
| age | 63.5 ± 12.5 | 71.9 ± NA | 59.3 ± 7.1 | 67.7 ± 5.7 | 64.5 ± 9.4 | < 0.001 |
we manually QC’d each modality for each of 500 baseline subjects
Summarize QC results for each modality
Summarize results for “joint” quality - how many subjects are high quality across all modalities at once?
| p | ||
|---|---|---|
| (N=549) | ||
| qcfail_orientation | ||
| p | ||
|---|---|---|
| (N=523) | ||
| qcfail_background_noise | ||
| p | ||
|---|---|---|
| (N=502) | ||
| NM_QC_Ratings_Z | < 0.001 | |
| p | |||
|---|---|---|---|
| (N=1204) | (N=1340) | ||
| qcfail_orientation | |||
| p | ||||
|---|---|---|---|---|
| (N=2969) | (N=187) | (N=854) | ||
| qcfail_orientation | ||||
| modality | Total | Successes | Failures | Percent_Success | Percent_Failure | failcat |
|---|---|---|---|---|---|---|
| T1w | 523 | 513 | 10 | 98.08795 | 1.9120459 | qchuman_T1w |
| T1w | 523 | 521 | 2 | 99.61759 | 0.3824092 | qcfail_background_noise |
| T1w | 523 | 519 | 4 | 99.23518 | 0.7648184 | qcfail_motion |
| T1w | 523 | 519 | 4 | 99.23518 | 0.7648184 | qcfail_other |
| modality | Total | Successes | Failures | Percent_Success | Percent_Failure | failcat |
|---|---|---|---|---|---|---|
| T2Flair | 549 | 524 | 25 | 95.44627 | 4.5537341 | qchuman_FLAIR |
| T2Flair | 549 | 548 | 1 | 99.81785 | 0.1821494 | qcfail_orientation |
| T2Flair | 549 | 545 | 4 | 99.27140 | 0.7285974 | qcfail_intensity_saturation |
| T2Flair | 549 | 543 | 6 | 98.90710 | 1.0928962 | qcfail_background_noise |
| T2Flair | 549 | 539 | 10 | 98.17851 | 1.8214936 | qcfail_motion |
| T2Flair | 549 | 547 | 2 | 99.63570 | 0.3642987 | qcfail_other |
| modality | Total | Successes | Failures | Percent_Success | Percent_Failure | failcat |
|---|---|---|---|---|---|---|
| DTIb0 | 1204 | 1110 | 94 | 92.19269 | 7.8073090 | qchuman_DTI |
| DTIdwi | 1340 | 1224 | 116 | 91.34328 | 8.6567164 | qchuman_DTI |
| DTIb0 | 1204 | 1188 | 16 | 98.67110 | 1.3289037 | qcfail_orientation |
| DTIdwi | 1340 | 1324 | 16 | 98.80597 | 1.1940299 | qcfail_orientation |
| DTIb0 | 1204 | 1198 | 6 | 99.50166 | 0.4983389 | qcfail_phantom |
| DTIdwi | 1340 | 1334 | 6 | 99.55224 | 0.4477612 | qcfail_phantom |
| DTIb0 | 1204 | 1204 | 0 | 100.00000 | 0.0000000 | qcfail_intensity_saturation |
| DTIdwi | 1340 | 1336 | 4 | 99.70149 | 0.2985075 | qcfail_intensity_saturation |
| DTIb0 | 1204 | 1154 | 50 | 95.84718 | 4.1528239 | qcfail_background_noise |
| DTIdwi | 1340 | 1286 | 54 | 95.97015 | 4.0298507 | qcfail_background_noise |
| DTIb0 | 1204 | 1150 | 54 | 95.51495 | 4.4850498 | qcfail_motion |
| DTIdwi | 1340 | 1270 | 70 | 94.77612 | 5.2238806 | qcfail_motion |
| DTIb0 | 1204 | 1190 | 14 | 98.83721 | 1.1627907 | qcfail_signal_dropout |
| DTIdwi | 1340 | 1318 | 22 | 98.35821 | 1.6417910 | qcfail_signal_dropout |
| DTIb0 | 1204 | 1188 | 16 | 98.67110 | 1.3289037 | qcfail_other |
| DTIdwi | 1340 | 1324 | 16 | 98.80597 | 1.1940299 | qcfail_other |
| modality | Total | Successes | Failures | Percent_Success | Percent_Failure | failcat |
|---|---|---|---|---|---|---|
| rsfMRI | 2969 | 2512 | 457 | 84.60761 | 15.3923880 | qchuman_rsfMRI |
| rsfMRI_LR | 187 | 168 | 19 | 89.83957 | 10.1604278 | qchuman_rsfMRI |
| rsfMRI_RL | 854 | 758 | 96 | 88.75878 | 11.2412178 | qchuman_rsfMRI |
| rsfMRI | 2969 | 2960 | 9 | 99.69687 | 0.3031324 | qcfail_orientation |
| rsfMRI_LR | 187 | 186 | 1 | 99.46524 | 0.5347594 | qcfail_orientation |
| rsfMRI_RL | 854 | 838 | 16 | 98.12646 | 1.8735363 | qcfail_orientation |
| rsfMRI | 2969 | 2955 | 14 | 99.52846 | 0.4715392 | qcfail_phantom |
| rsfMRI_LR | 187 | 187 | 0 | 100.00000 | 0.0000000 | qcfail_phantom |
| rsfMRI_RL | 854 | 854 | 0 | 100.00000 | 0.0000000 | qcfail_phantom |
| rsfMRI | 2969 | 2955 | 14 | 99.52846 | 0.4715392 | qcfail_spacing |
| rsfMRI_LR | 187 | 187 | 0 | 100.00000 | 0.0000000 | qcfail_spacing |
| rsfMRI_RL | 854 | 854 | 0 | 100.00000 | 0.0000000 | qcfail_spacing |
| rsfMRI | 2969 | 2949 | 20 | 99.32637 | 0.6736275 | qcfail_intensity_saturation |
| rsfMRI_LR | 187 | 177 | 10 | 94.65241 | 5.3475936 | qcfail_intensity_saturation |
| rsfMRI_RL | 854 | 822 | 32 | 96.25293 | 3.7470726 | qcfail_intensity_saturation |
| rsfMRI | 2969 | 2388 | 581 | 80.43112 | 19.5688784 | qcfail_motion |
| rsfMRI_LR | 187 | 187 | 0 | 100.00000 | 0.0000000 | qcfail_motion |
| rsfMRI_RL | 854 | 854 | 0 | 100.00000 | 0.0000000 | qcfail_motion |
| rsfMRI | 2969 | 2804 | 165 | 94.44257 | 5.5574267 | qcfail_signal_dropout |
| rsfMRI_LR | 187 | 172 | 15 | 91.97861 | 8.0213904 | qcfail_signal_dropout |
| rsfMRI_RL | 854 | 806 | 48 | 94.37939 | 5.6206089 | qcfail_signal_dropout |
| rsfMRI | 2969 | 2968 | 1 | 99.96632 | 0.0336814 | qcfail_temporal_noise |
| rsfMRI_LR | 187 | 186 | 1 | 99.46524 | 0.5347594 | qcfail_temporal_noise |
| rsfMRI_RL | 854 | 846 | 8 | 99.06323 | 0.9367681 | qcfail_temporal_noise |
| rsfMRI | 2969 | 2957 | 12 | 99.59582 | 0.4041765 | qcfail_insufficient_volumes |
| rsfMRI_LR | 187 | 187 | 0 | 100.00000 | 0.0000000 | qcfail_insufficient_volumes |
| rsfMRI_RL | 854 | 854 | 0 | 100.00000 | 0.0000000 | qcfail_insufficient_volumes |
| rsfMRI | 2969 | 2968 | 1 | 99.96632 | 0.0336814 | qcfail_FOV |
| rsfMRI_LR | 187 | 187 | 0 | 100.00000 | 0.0000000 | qcfail_FOV |
| rsfMRI_RL | 854 | 854 | 0 | 100.00000 | 0.0000000 | qcfail_FOV |
| NM.QC.cat | Total | Successes | Failures | Percent_Success | Percent_Failure | failcat |
|---|---|---|---|---|---|---|
| overall | 478 | 386 | 92 | 80.75314 | 19.246862 | NM_QC_Ratings_Failures |
| z.coord | 478 | 391 | 87 | 81.79916 | 18.200837 | NM_QC_Ratings_Z |
| sn.vol | 478 | 471 | 7 | 98.53556 | 1.464435 | NM_QC_Ratings_SNVol |
| avg.int | 478 | 473 | 5 | 98.95397 | 1.046025 | NM_QC_Ratings_AvgIntensity |
| sd.int | 478 | 473 | 5 | 98.95397 | 1.046025 | NM_QC_Ratings_SDIntensity |
| max.int | 478 | 473 | 5 | 98.95397 | 1.046025 | NM_QC_Ratings_MaxIntensity |
for each, summarize how processing improves the data
rsfMRI is “easy” to see - throw away motion corrupted data
developed methodology specific for PPMI that was informed by this process
automated QC for these modalities for future work
## [1] " <0> BUILD-V <0> BUILD-V <0> BUILD-V <0> BUILD-V <0> "
## [1] "initialDataTerm: -0.375449406559412 <o> mixer: pca <o> E: cca"
## [1] "Iteration: 1 bestEv: -0.0317598345806564 bestIt: 1 CE: -0.0317598345806564"
## [1] "Iteration: 2 bestEv: -0.127153827109486 bestIt: 2 CE: -0.127153827109486"
## [1] "Iteration: 3 bestEv: -0.159247716566849 bestIt: 3 CE: -0.159247716566849"
## [1] "Iteration: 4 bestEv: -0.169491210153665 bestIt: 4 CE: -0.169491210153665"
## [1] "Iteration: 5 bestEv: -0.174768298475065 bestIt: 5 CE: -0.174768298475065"
## [1] "Iteration: 6 bestEv: -0.174768298475065 bestIt: 5 CE: -0.152803217187931"
## [1] "Iteration: 7 bestEv: -0.199710965219991 bestIt: 7 CE: -0.199710965219991"
## [1] "Iteration: 8 bestEv: -0.199710965219991 bestIt: 7 CE: -0.19579644862074"
## [1] "Iteration: 9 bestEv: -0.199710965219991 bestIt: 7 CE: -0.19579644862074"
## [1] "Iteration: 10 bestEv: -0.199710965219991 bestIt: 7 CE: -0.19579644862074"
## [1] "Iteration: 11 bestEv: -0.199710965219991 bestIt: 7 CE: -0.19579644862074"
## [1] "Iteration: 12 bestEv: -0.199710965219991 bestIt: 7 CE: -0.196780449833828"
## [1] "Iteration: 13 bestEv: -0.208143705672403 bestIt: 13 CE: -0.208143705672403"
## [1] "Iteration: 14 bestEv: -0.208143705672403 bestIt: 13 CE: -0.205026650021558"
## [1] "Iteration: 15 bestEv: -0.239617756820512 bestIt: 15 CE: -0.239617756820512"
## [1] "Iteration: 16 bestEv: -0.239617756820512 bestIt: 15 CE: -0.239617756820512"
## [1] "Iteration: 17 bestEv: -0.239617756820512 bestIt: 15 CE: -0.239617756820512"
## [1] "Iteration: 18 bestEv: -0.239617756820512 bestIt: 15 CE: -0.239617756820512"
## [1] "Iteration: 19 bestEv: -0.239617756820512 bestIt: 15 CE: -0.239617756820512"
## [1] "Iteration: 20 bestEv: -0.244185533257034 bestIt: 20 CE: -0.244185533257034"
## [1] "Iteration: 21 bestEv: -0.244185533264272 bestIt: 21 CE: -0.244185533264272"
## [1] "Iteration: 22 bestEv: -0.244185533264272 bestIt: 21 CE: -0.244185533264272"
## [1] "Iteration: 23 bestEv: -0.244185533264272 bestIt: 21 CE: -0.244185533264272"
## [1] "Iteration: 24 bestEv: -0.244185533264272 bestIt: 21 CE: -0.244185533264272"
## [1] "Iteration: 25 bestEv: -0.244185533264272 bestIt: 21 CE: -0.244185533264272"
## [1] "Iteration: 26 bestEv: -0.244185533264272 bestIt: 21 CE: -0.244185533264272"
## [1] "Iteration: 27 bestEv: -0.244185533264272 bestIt: 21 CE: -0.244185533264272"
## [1] 0.06308406
## [1] 0.03358377
## [1] 0.02338849
## [1] "1 0.000182813961727995"
## [1] 0.03914674
## [1] 0.0688519
## [1] 0.02239701
## [1] "1 0.253184809658498"
## [1] 0.04997151
## [1] 0.06109953
## [1] 0.01932642
## [1] "1 0.117369987110731"
## [1] 0.0155129
## [1] 0.09418174
## [1] 0.03088837
## [1] "1 0.0772706971826326"
## [1] 0.0116334
## [1] 0.07219495
## [1] 0.02426768
## [1] "1 0.0245371892594724"
## [1] 0.02914037
## [1] 0.07281831
## [1] 0.02122702
## [1] "1 0.00413298641191162"
## [1] 0.04614809
## [1] 0.04694281
## [1] 0.05216954
## [1] "1 0.444470145275164"
## [1] 0.06048338
## [1] 0.0978308
## [1] 0.01358586
## [1] "1 0.116907067117507"
## [1] 0.07213305
## [1] 0.0697766
## [1] 0.0292922
## [1] "1 0.0735499924260931"
## [1] 0.05090588
## [1] 0.1141006
## [1] 0.01063304
## [1] "1 0.168098675956571"
## [1] 0.02541644
## [1] 0.09387618
## [1] 0.03808487
## [1] "1 0.201933845040915"
## [1] 0.05573603
## [1] 0.09141141
## [1] 0.04255581
## [1] "1 0.01919865559824"
## [1] 0.04590256
## [1] 0.06853646
## [1] 0.02107676
## [1] "1 0.42165835391733"
## [1] 0.05111144
## [1] 0.04577096
## [1] 0.01435047
## [1] "1 0.0708125993718408"
## [1] 0.05406733
## [1] 0.05454088
## [1] 0.01260812
## [1] "1 0.212117989047716"
## [1] 0.02707338
## [1] 0.06951353
## [1] 0.03713177
## [1] "1 0.838761144764153"
## [1] 0.0463601
## [1] 0.01240573
## [1] 0.01356592
## [1] "1 0.147990214086151"
## [1] 0.03481475
## [1] 0.1041289
## [1] 0.02005062
## [1] "1 0.0918154379513776"
## [1] 0.02238462
## [1] 0.05018068
## [1] 0.04757634
## [1] "1 0.0841661059386939"
## [1] 0.03900399
## [1] 0.05628325
## [1] 0.03711137
## [1] "1 0.330606061335976"
## [1] 0.04638839
## [1] 0.04219776
## [1] 0.02991352
## [1] "1 0.180275099839925"
| permstate | anv | pvalt1 | pvalrs | pvaldt |
|---|---|---|---|---|
| original | 0.0001828 | 0.0024539 | 0.0869260 | 0.0035658 |
| perm | 0.2531848 | 0.1140175 | 0.1861437 | 0.7066202 |
| perm | 0.1173700 | 0.1843373 | 0.3435084 | 0.0409037 |
| perm | 0.0772707 | 0.6311595 | 0.5723706 | 0.0103567 |
| perm | 0.0245372 | 0.1506731 | 0.1488727 | 0.0217073 |
| perm | 0.0041330 | 0.6336158 | 0.1130059 | 0.0017395 |
| perm | 0.4444701 | 0.2083604 | 0.3016553 | 0.8922591 |
| perm | 0.1169071 | 0.0584459 | 0.2420679 | 0.1599787 |
| perm | 0.0735500 | 0.4102780 | 0.9401177 | 0.0123246 |
| perm | 0.1680987 | 0.9676549 | 0.6917760 | 0.0314073 |
| perm | 0.2019338 | 0.1558100 | 0.5523330 | 0.1498735 |
| perm | 0.0191987 | 0.7096165 | 0.8340096 | 0.0017361 |
| perm | 0.4216584 | 0.3479091 | 0.1755137 | 0.9776325 |
| perm | 0.0708126 | 0.6606754 | 0.7714953 | 0.0112521 |
| perm | 0.2121180 | 0.8806537 | 0.7828186 | 0.0378230 |
| perm | 0.8387611 | 0.7557867 | 0.7384605 | 0.4004283 |
| perm | 0.1479902 | 0.6723744 | 0.2996158 | 0.0424704 |
| perm | 0.0918154 | 0.9887208 | 0.6955649 | 0.0145325 |
| perm | 0.0841661 | 0.1468834 | 0.5218746 | 0.0270130 |
| perm | 0.3306061 | 0.1082332 | 0.4396962 | 0.8688471 |
| perm | 0.1802751 | 0.1296333 | 0.4942471 | 0.1411465 |
## Warning: Using `bins = 30` by default. Pick better value with the argument
## `bins`.
## Warning: Groups with fewer than two data points have been dropped.
## Warning in max(ids, na.rm = TRUE): no non-missing arguments to max; returning
## -Inf